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ABSTRACT 



A method of forming a template of an image of an object 
includes the steps of: detecting occurrences of at least one 
feature type within the image which meet a respective 
criterion; for each such occurrence, determining a feature 
position which is a point within the image at which the 
occurrence of the feature type is to be considered as being 
located; and constructing a structural mesh of links between 
the feature positions. A method is also disclosed of forming 
a template derived firom a plurality of templates formed £rom 
respective images of different members of the class of 
objects and combining the templates to form a generalized 
template. In particular the templates are combined using a 
genetic algorithm. The invention provides a method of 
forming a template from one or more images which does not 
rely on a priori assumptions about the salient features of the 
object for which a template is to be obtained. 

10 Claims, 6 Drawing Sheets 
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METHOD OF FORMING A TEMPLATE OF priori information, it is important that the information incor- 

AN IMAGE OF AN OBJECT FOR USE IN porated into the template is accurate. For example, a sim- 

THE RECOGNITION OF THE OBJECT plification of an edge so that it can be mirrored in the 

template as a single geometric line may in fact hinder 

FIELD OF THE INVENTIGN 5 matching of the template to an image. It has been found, for 

. example, that the edge of the lower eyelid can vary consid- 

This mvention relates to a method of formmg a template ^^^^^^ between the populations of different countries so that 

of an image of an object, which template finds particular, but ^ template designed to recognise a typically "Western" eye 

not exclusive, appUcation to the recognition of objects in ^^^^ y^^^^ ^^^^ is attempted to fit the template to a 

images for security and surveillance appUcations and in typically "Eastern" eye. 

model-based image coding. ^ ^ characteristic of such a known template that it 

BACKGROUND OF THE INVENTION ^ some cases, not describe accurately the feature in an 

image but rather what the human perceives to be the saUent 

A template is a representation of an image of an object features of the object in the image. A further disadvantage is 

which aims to contain the image's saUent features so that the 15 that these templates have to be designed "by hand" and 

template can be subsequently compared to other images, the individually coded which is time consuming and may be 

goodness of fit of the template to another image providing a carried out inaccurately, 
measure of the likelihood that this image is of or contains a 

sub-image of the object. For example, a series of templates DESCRIPTION OF PREFERRED 

may be formed from respective images of the faces of a 20 EMBODIMENTS 

group of people. An unage of a peison's &ce captured by a According to the present invention a method of forming 

secunty cainera could then be compared to the templates to ^ ^^^^ ^^^.^^ j^^j^^^^ 

determine if that person is one of the group. > , . ^ , ^ • l • 

, , . , _ J. ajdetectuigoccurrencesof at least one feature type within 

Aknown approach to the use of templates is descnbed in ^ ^j^j^^ ^^^^ ^ respective criterion; 

an article entitled Feature Extraction From Faces Usmg . . . .. r 

Deformable Templates" by A L -ftiille. D S Cohen and P W >') !" ^"«=*' occurrence determmmg a feamre posihon 

HaUinan. Harvard Robotics Laboratory Technical Report '^^'^^ » PT ' ^ T^" 

33-2, 1988. m this approach a feamre or object of interest is [^"'^^ f ^^"'"'^ 'yP^ '° bei°g 

described by a geometric representation which is specified ^ ' ^ 

by a set of parameters which incorporate knowledge of the 30 c) constructing a structural mesh of links between the 

shape of the object. The article gives examples of two such feature positions. 

templates; one for use in recognising an eye and the other a The method of the present invention processes an image 

mouth. In the former case the eye template decided upon by form a template appropriate to a particular object (for 

the authors incorporated a circle corresponding to the outer example an eye or a face) directly from a traming image of 

boundary of the iris, boundary contours corresponding to the ^5 the object. The advantage of this method is that the template 

top and bottom of the eye comprising two parabolas, regions encodes the significant attributes of the object which are 

corresponding to the whites of the eyes, and two points present in the image and does not attempt to encode possibly 

corresponding to the centres of the whites of the eyes. illusory details. Furthermore, the method can be carried out 

The template of YuiUe et al is deformable in that the without any human input into the design of the templa^^^ 

- ^, ^ , ^ i_- * * ^. , 40 can potentially be applied to any obiect presented in a 

components of the template can move subject to notional . r . ^ ^ y j .1. . 1 . 1. j • 

, .--r L*. *t. .Ai/u trainmg image. Once formed the template can be used in the 

structural, restrammg forces between the components. When r 1 j * ui . % . r • • 

. iV . * L J . . * . * f maimer of known deformable templates for use m image 

the template is matched to a test unage, the components of .^.^ ^ ]ic&\i ns 

the template are attracted to various valleys, edge and peak recogni ion an o er app ca ons. 

* f *!. u- u Vf .V The template is derived, m the method of the invention, 

components of the image which attractive forces are gener- i-*^ ..... • ... .■, r 

11 ■ * J L * ^ 1 r u * *u .45 directly from the traimng image with no guidance from a 

ally resisted by the structural forces between the components , ^ ^ ..f . . , , j j: 1. . .1. 

ofihetemplate-Anenergyfunctionisdefinedintermsofthe l'™^" °P«"'°' and without prior knowledge of what the 

attraction of the template components to the features of the ^"8"' ^^P"^"** '^""8 necessary by processmg a traimng 

image and the structural forces resisting deformation of the i .j . a .u j r *u 

, , . . 1 . u • AC J * • • - 4U- In order to provide the method of the present mvention 

template, the template bemg deformed to mmmiise this „ ..u • ■ % ui * i * 

energy function. This energy fiinction gives a measure of the '° ' reasonable opportunity of deriving a stuuble template 

goodness of fit of the template to the iiage and can be used *° fjf P'^^^^^y withm he 

Is a measure of whether the template has, in fact, been ^^age and « separated f^om other features, that is. only he 

successfully matched to the image, ie that the image does ?bject for which a template is sought should be present m the 

. , J ^ J * *i. * 1 r image. The conditions may be relaxed when the template is 

mdeed correspond to the template. , ^ , - f u- * * . • 

*^ ^ . ^ , , . J 55 subsequently used for matching to a test unage. 

This approach to the formmg of templates was devised so jhe nodes of the structural mesh of links between the 

that a priori information avaUable about the appearance of ^^^^^ positions can be considered as linked tokens in 

the features of the object to which the template was to be ^^^^ ^^^^ ^^^^^ ^^^^ ^ ^^^^^^ p^^^i^ly 

matched, m Yuille et al's particular case facial features, attribute values such as the strength and the extent of the 

could be incorporated into the template so providing a ^^^^^ ^^^^^ ^h^ ^^^^^^^ ^^^^ by the 

mechanism for detectmg edges or other features which method of the present invention can be considered as being 

comprise local information as global structures. constructed of a network of such primitive tokens that have 

BRIEF DESCRIPTION OF TOE INVENTION ^""^^"^^ image by appropriate image 

processing operations. 

The Apphcant has appreciated that there are, however, 65 The method of the present invention is selective as to 

disadvantages associated with this approach to forming a which occurrences of a feature type are included in the 

template. Firstly, because the template design is based on a template. For example, it is preferable that only feature types 
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having sufficient strength in the image are considered as A token associated with any feature type may include as 

these are more likely to be salient image attributes so the additional attributes the size, principal orientation and aspect 

criterion can be set to discard all but the desired featiu-e types ratio of the activity area, for example, 

detected. It is possible to extract meaning from the final template. 

In general, a feature type will extend over a portion of the 5 jhe template which has been derived can be displayed by 
image. In the template of the present invention, the feature individually interpreting each constituent token. As each 
type is considered to be located at a point within the image, ^^^^^^ describes an expected image attribute, the correspond- 
so that it can form a node of the structural mesh of the attribute can be displayed in the appropriate position on 
template. Thus, for an occurrence of a feature type meetmg ^ ^^^^ template to be viewed, as a 
the required criterion a point has to be determmed which is ^^^^ ^^^^^^^ ^ ^^^^ -^^^ ^^^^ 
to be that point where the feature is deemed to be located. ' .u- ♦-«,«iot- k« .,L»»i uk-ikw,^ 
This could, for example, be the centroid of the feature type. parts of the template by visual ii^pecUon and labelling 

Tlie structural mesh of links between the feature positions ^7 P!^?°^ "f'^^ ^ not themselves actually be 

will incorporate structural forces between the nodes of the coincident with any of the tokens. For example, it may be 

mesh to restrain the deformation of the template when being appropnate that a key pomt is maintained with respect to an 

matched to test images. image to which the template is bemg fitted, for example for 

The detection of an occurrence of a feature type within an a face the nominal centre position of a nose could be labelled 

image can be carried out directly by: which may not have any coincident image activity. This 

d) forming a representation image in which only those position could then be obtained by its context within the fit 
portions of the image which are of the feature type are of the template to any face. 

enhanced; 20 A template formed by analyzing a single training image 

e) thresholding the representation image to obtain an may be particularly suited to identifying when that image 
image bitmap; re-occurs by matching the template to it. However, there are 

0 locating areas of activity within the image bitmap; and many applications where it is necessary to identify the 

g) determining which of the located areas of activity meet presence of an object which has not appeared specifically in 

the criterion to be detected as an occurrence of the 25 a training image. For example, if it is necessary to recognise, 

feature type. in general terms, the presence of an eye in an image, then a 

These steps are carried out for each class and orientation template formed from analyzing one example of an eye may 

of feature type that is to be used to form a template. Once not be ideally suited to this general recognition problem as 

all the feature types have been identified then the structural the particular eye on which the template was formed may 

mesh of links can be formed between them. 30 contain feature types which are not a common characteristic 

The feature types may include, for example, oriented of eyes generally. In these circumstances it has been found 

edge, valley and peak images which may not be ideal but beneficial to form a generalised template formed by com- 

which have been found to provide a useful starting set for bining templates formed from several images each being a 

forming templates according to the present invention. different example of the type of object to be recognised. So, 

The chosen feature types can be identified by any appro- 35 for example, a series of different eyes can be used to form 
priate means. For example, the edge images may be found a respective series of templates. These templates are then 
by using simple oriented Sobel filters and the peak and combined to form a generalised template, 
valley images derived using orientation-sensitive morpho- A number of possible approaches to the problem of 
logical filters. Preferably to remove any overlap of responses combining the templates to form a composite template have 
between feature types of a given class of feature (eg edges 40 been considered by the applicant. These included a graph 
or valleys) but of different orientations, only the maximum matching approach, for example as described by J Segan in 
valued picture elements across all orientations are retained; an article entitled "Model Learning and Recognition of 
the corresponding picture elements at all other orientations No n- Rig id Objects" Computer Vision and Pattern 
are set to 0. It is possible in this way to obtain a set of Recognition, pages 597-602, 1989; and an energy minimi- 
representation images for each orientation of each feature 45 sation comparison technique, similar to that used in fitting 
type. templates to images, but used between templates. 

The criterion that a feature type is to be regarded as However, the approach which appears to be most prom- 
occurring in an image is conveniently that the number of ising at the time of filing this application is to use a genetic 
picture elements in an area of activity is greater than a algorithm. Genetic algorithms are based on the principle of 
threshold size. Those areas that are smaller are discarded, 50 natural selection and evolution found in biological systems, 
only the remainder being used to form the nodes of the The ^plicant has devised and applied a genetic-type algo- 
structural mesh of the template. rithm to combine templates. A set of possible solutions, in 

The feature position of an occurrence of a feature type is this case templates, are maintained as a population which is 
preferably regarded as being the centroid of the correspond- adapted by combining the best members whilst others are 
ing area of image activity, or blob, in the modified image 55 discarded to generate increasingly successful templates bet- 
bitmap, ter able to fit faces, for example. 

The present invention is not restricted to any particular An embodiment of the present invention will now be 

type of structural mesh formed to interlink the feature types. described, by way of example, only with reference to the 

One convenient structural mesh is formed by performing a accompanying drawings in which: 

known Delauney triangulation of all the feature types. The 60 FIG. 1 is a photograph constituting an image of a man's 

resulting triangulated mesh of feature types, or tokens, then head and shoulders; 

forms the template for the object represented in the training FIGS. 2{a) to 2{d) are photographs of representation 

image. images obtained from the image of FIG. 1, each of which 

In some instances it may be preferable to delete from the retains occurrences of a different feature type; 

template links greater than a predetermined length in order 65 FIGS. 3(a) to 3(d) are photographs of image bitmaps 

to allow greater freedom of movement of the template obtained fi-om the representations images 2(a) to 2(d), 

between areas which may not be closely linked physically. respectively; 
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FIGS. 4(a) to 4(d) are photographs of those areas of this feature was a token which represented the feature type 

activity larger than a predetermined threshold size and so are and its position within the image, 

regarded as occurrences of a feauire type in the image Any attributes that may be of interest according to the 

bitmaps of the FIGS. 3(fl) to 3(d), respectively; particular application to which the template is going to be 

HG. 5 is a photograph of a template formed from the 5 P^t could also form part of the token. For example, it could 

feature types of HGS. 4(a) to 4(d) shown superimposed on contain information about the size of the feature and its 

the image of ¥IG, 1 from which the template was derived; strength. . r . , u u 

HG. 6 is a schematic diagram showing a chromosomal k^^'"?^' 1 f P^^,^"^^ was a senes of tokens which 

r . 1. r5 ^«^*;^«n,,Unc^^ t»«,«Uf« characterised the feature type content of the ongmal image. 

representauon of a template for a geneUcally based, template ^ ^^^^ ^^^^^ ^ J ^^^^ obtained for the other 

combimng algonthm; ^ . ^ ^° feahire type that were to be used in forming the final 

HG, 7 IS a diagram of an unage divided mto zones for template. The full set of tokens for all the feature types was 

representation as a chromosome; ^hen constructed which set formed the basis of a template of 

FIG, 8 is a representation of the information associated original image obtained by triangulating the feature type 

with each gene of the chromosome of FIG. 6; positions. 

FIG. 9 is a diagram showing a method of dividing an is Referring now to FIG. 5 there is illustrated a template 

image into zones by a recursive method; constructed by forming a structural mesh of links between 

FIG. 10 is a graph showing the subdivision recursion tree the feature positions of all the tokenised feature types of 

associated with FIG. 9; FIGS. 4(a) to 4(d). In this case the mesh was formed by 

FIGS. 11 and 12 are diagrams showing how two parent connecting the feature positions with a triangulated mesh 

chromosomes produce two child offspring chromosomes by 20 using Delauney triangulation of all the feature positions. The 

a crossover mechanism; and template so formed was the desired template representation 

FIG. 13 is a diagram showing how a chromosome can be of the original image, 

modified by inverting a gene string. The photograph at FIG. 5 shows the template superim- 

Referring first to FIG. 1 there is shown an exemplary Posed on the original image with the feature positions being 

image of a man's head and shoulders for which a method of 25 denoted by node positions which are linked by straight line 

forming a template according to the present invention will segments. .... 

u^A^c^^iu^A ^f^u^^^A K« m**o«c /^f a The choicc of which feature types are to be used m the 

now be described. The image was captured by means oi a _ . _ ^ . . j rm. * i * j u u 

T\7 « A^^ri^ ^^rUir^h ^tr^r-^A formation of a template is not fixed. The set selected should, 

TV camera coupled to a image stonng device which stored , u u *u * •* u j- *- * • * 

. . - , *^ 1 * I however, be chosen so that its members are distmct, mror- 

Uie values of the coi^ti uen picture elements. In this par- ^^^^^ ^ ^^^^ ^ ^^^^^^ ^^^^^ ^ 

ticular example four feanire types were considered, namely 30 independent of chang^ in iUumination as possible. An 
vertical and horizontal edges and edges rumiing at 45 to the ^^^^^ ^^^.^j^^ «Pre-Attentive Processing in Vision" by Anne 
vertical. The image was therefore processed m known Treisman, in Computer Vision, Graphics, and Image Pro- 
manner to form four representation images in which only cessing vol. 31, 156-177 (1985) defines a possible set of 
those portions of the image of a respective feature type were sm-h feature types. 

retained. The resulting representation images for these fea- 35 in general, the different processed images will not be 

tures are shown in FIGS. 2(a) to 2(d). shown visually on a visual display unit during processing of 

Consideration will now be restricted to the processing of an image to form a template, rather the steps of the invention 

the FIG. 2(a) representation image to describe how feature will utilise stored representations of the training image and 

types were extracted from a processed representation. The its processed forms. 

same process was also applied to the other three represen- 40 A template according to the present invention can be used 

tation images of FIGS. 2(b) to 2(d). to both recognise and describe the shape of novel instances 

First, the representation image of FIG. 2(a) was thresh- of the object type upon which it has been trained by 

olded to obtain an image bitmap as illustrated in FIG. 3(a) initialising it in the vicinity of an object in an image and 

which retained only those portions of the image appropriate allowing template feature types to be attracted to corre- 

fealure type. The picture elements of the image bitmaps of 45 spending characteristics in the target image whilst at the 

different orientations of a given feature type were then same time deforming its geometric shape within limits 

compared and the maximum valued picture element only according to how closely image detail can be matched. This 

retained. That is the image bitmap containing the maximum fitting process is analogous to that used by traditional 

valued picture element retained that pixel whereas the deformable templates of the type described by Yuille et al, 

equivalent picture elements in the other image bitmaps with 50 referenced above. If a template is to be suitable for fitting 

which it has been compared are deleted. This provides a and so describes the shape of the majority of novel instances 

series of image bitmaps for a given feature type without any of a given object which may be subsequently encountered 

overlap between the areas of activity for the various orien- then it has been found by the Applicant to be greatly 

tations of that feature type. The size of the individual areas beneficial that templates are formed from more than one 

of the image bitmap of FIG. 3(a) were then sized by 55 image of different members of the class of object which is 

counting the number of picture elements in the particular to be recognised and combimng the templates to form a 

feature. This gave a measure of the size of the feature which generalised template. This scheme, which also constitutes an 

was then tested against a threshold. Those which were too aspect of the present invention, provides a generalised 

small were deleted from the image bitmap to obtain the template which incorporates the salient feamres appropriate 

modified image bitmap of FIG. 4(a). 60 to the class of object rather than the specific features for a 

The areas of activity that remained in the modified image particular member of the class. This provides a template 

bitmap of FIG. 4(a) were then detected as an occurrence of which is more suitable for classifying objects as opposed to 

the feature type, all the other instances being ignored. The identifying a particular member of a class of objects where 

feature position was calculated by weight averaging the atemplatesolely based on that one image will be preferable, 

picture element constituents of the feature, ie the centroid of 65 There will now be described a particular method of 

the feature, which position was considered to be the point of combining templates to form such a generalised template 

the image at which the feature was located. Associated with according to the present invention. 
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The method of forming a generalised template presently 
preferred by the Applicant is to use a genetic algorithm to 
determine which of the template features are most valuable 
in recognising an object in the class on which it is trained. 
The basic algorithm requires a genetic representation analo- 
gous to the chromosomes in living creatures, a way of 
interpreting this representation to form a population member 
and a method of establishing the fitness of each population 
member. 

The algorithm itself starts with an initial, possibly poor or 
unfit, population and iteratively improves the fitness by 
preferentially selecting fit members to be the parents of the 
next generation's members. Fitter members thus tend to 
contribute preferentially to the gene pool of the next 
generation, and it can be shown that genes which contribute 
to the above average fitness tend to increase exponentially in 
frequency from generation to generation. 

The power of genetic algorithms does not lie merely in the 
survival of the fittest strategy but also in the way in which 
the two population members are combined. The chromo- 
somes from any two parents should both contribute to the 
chromosome of their children (usually two are produced) by 
means of crossover. This is a phenomenon observed in 
animal genetics in which two chromosomes align with one 
another and contiguous strands of genetic material are 
exchanged. Thus new combinations of genes from fitter 
population members are constantly being tried together to 
obtain an overall fitness improvement. 

In the context of the present invention, the set of templates 
to be combined should be based on a set of training images 
which are representative of objects that are likely to be 
encountered in the future. In particular, the set of image 
characteristics described by the derived templates should 
contain all the relevant elements to describe the appearance 
of most new objects likely to be presented to an image 
recognising apparatus. 

The genetic algorithm is then applied to swap elements 
between chromosomes from generation to generation which 
has the effect of producing new chromosomes representing 
templates which contain new mixtures of element taken 
from various different initial templates. A part of a template 
good at matching for example eyes in one template may be 
co-joined with a part of another template good at matching 
a mouth thereby producing a composite template good at 
doing both tasks. 

The details of an implementation of such a genetic 
algorithm will now be described in more detail. 

Each template that is going to contribute to the composite 
template must be encoded as a chromosomal representation 
for the genetic algorithm to operate upon . 

As well as a chromosomal representation of a template 
there mtist also be a method of interpreting chromosomes in 
order to derive the corresponding template individual. In the 
context of the present invention one must also be able to 
create a chromosome given the template description in order 
to initialise the starting population given the training set of 
derived templates. 

There are certain properties that it is desirable a chromo- 
some representation exhibits due to constraints laid down by 
genetic algorithms in general as well as practical consider- 
ations. It is preferable to have a fixed length chromosome 
description consisting of a fixed number of gene units as this 
helps during mating and crossover of chromosomes. Col- 
lections of genes which provide advantageous attributes in 
individuals are said to be co-adapted, that is they work well 
together. It is preferable that such co-adapted genes congre- 
gate locally on the chromosome such that they tend to be 
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preserved during crossover. Each gene should preferably 
represent an element from an alphabet of low order, ie it 
should only be able to take on a small range of values. 
The chromosomes described to date in the scientific 

5 literature consist of linear strings of genes. Although a 2-D 
chromosomal representation may seem to be closer to the 
2-D nature of the templates, the present embodiment uses 
the more traditional linear mapping of genes for which the 
theory is now reasonably well developed albeit in other 

10 applications. Alternative chromosome representation strate- 
gies may, of course, be adopted if they prove to be advan- 
tageous. The linear structure of the genes within a chromo- 
some should not present any problems, however, since 
during initialisation the feature types which are in close 

15 proximity spatially are represented by genes close together 
on the chromosome. The property exhibited by genetic 
algorithms is to provide clustering of co-adapted genes on 
the chromosome which should help to prevent such clusters 
firom being broken up during crossover. 

20 Referring now to FIG. 6 a chromosome representation of 
a template used comprised a set of n genes G,-, i='0 to n-1, 
each gene G, representing the feature type in one area of the 
image. In order to fix the length of the chromosome the 
image area was quantised into a fixed number of zones 

25 (square regions) Z,- as shown in FIG. 7. Each zone Z,- could 
contain one or more instances of the recognisable feature 
types but only one was coded in the chromosome. 

FIG. 8 shows the information encoded by a gene G^ of a 
chromosome. It consisted of three gene fields, Fj, F2 and F3. 

30 Gene field F^ contained the number of the zone Z,. which it 
encodes — in this case zone number 34. Gene field F2 con- 
tained a flag indicating whether the feature type within that 
zone is active. Gene field F3 denoted the feature type — ^in 
this case a top right to bottom left, 45° edge. 

35 Since the genes G^- contained a reference to the zone Z^- of 
the image to which they corresponded the zone number 2.^ 
and the position of a gene on a chromosome were therefore 
independent so they could change their relative chromosome 
location freely, as desired. The genes were restricted to 

40 elements from a relatively low order alphabet consisting of 
the number of allowable feature types. 

As stated above, the zone Z,- to which a gene G,. corre- 
sponded was indicated by a zone ntimber. The actual zone 
details such as zone position and adjacency information 

45 were held in a separate location in order to minimise the 
extraneous detail which bad to be coded by each gene. A 
compact representation is more efficient which was impor- 
tant during implementation of the algorithm. 
The connectivity of the feature types within the template 

50 structural mesh was not encoded at all within the genetic 
representation but was created automatically during expres- 
sion of the chromosome to produce the corresponding 
template as will be described below. 

Although genetic algorithms are often initialised ran- 

55 domly and then allowed to search for a solution, it is 
preferable to initialise the population from a set of base 
templates derived from a set of training images of the object 
of interest. It should be noted that these images were very 
crudely normalised such that the centres of the eye and 

60 mouth in each image also align across all the images in order 
to assist the genetic algorithm in combining templates 
during later stages. 

The first stage of initialisation was to convert the base 
templates into their equivalent chromosome representations. 

65 This consisted of determining within which zone each 
template feature type lay and then assigning a gene for that 
feature type. When more than one 9 feature type falls within 
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a single zone, an arbitrary one was kept the rest being 
discarded. The only remaining variable in the initialisation 
was the ordering of the genes on the chromosome. Since it 
was not known at initialisation which feature types were 
related, and hence which genes were co-adapted, the optimal 5 
ordering of genes was not known. However, a strategy was 
adopted which located genes containing spatially close 
tokens in similar proximity on the chromosome as it had 
been found that local feature types are often related to one 
another. lO 

The strategy used in the present embodiment to decide 
which genes go where on the chromosome consisted of a 
recursive spatial subdivision process applied to the space of 
the entire image. 

Referring to FIGS. 9 and 10, a full image 62 was 15 
recursively split once, evenly, into four sub-regions to S4 
about the midpoint of the image, which sub-regions were 
then randomly ordered and split again until the sub-regions 
S^yjt reached the specified zone size. Each lime the recursion 
reached a minimum sized zone sub-region the corresponding 
gene was added onto the end of the growing list of genes on 
the chromosome. Spatially proximate zones were generally 
close in the recursion tree and so mapped onto nearby genes. 

Although there is no perfect mapping from 2-D to 1-D 
which preserves locality, this process provided orderings 
which exhibited a high degree of locality whilst introducing 
some variability into the ordering as between the different 
initial chromosomes. 

The chromosomal description of an individual is known 
as the genotype and this can be expressed or interpreted to 
form the individual itself known as the phenotype. It is 
necessary to be able to express the chromosome in terms of 
the phenotype before one can go on to evaluate the fitness of 
the given individual. The phenotype consists of a template in 
the present instance. 

The process of expressing the chromosome was relatively 
straightforward in the present case. Each gene of the 
chromosome was examined and if marked as active a 
corresponding feature type was created in the generalised 
template structure of the type specified by the field F3 of the 
gene G,-. The gene G, also implicitly codes the position of the 
feature type in terms of the zone index which could be cross 
referenced to the list of zone details to provide the feature 
position information. 

The template nodes had to be triangulated as previously 
described to form the template and for fitting to a given test 
image. The triangulation was performed as a post- 
processing stage just prior to fitting the template. No node 
connectivity information was encoded within the genetic 
representation itself. 

In order that the mechanism of selection could proceed it 
was necessary to have a means of evaluating a measure of 
merit or fitness for a given template offspring arising out of 
the combination of two template chromosomes. The indi- 
vidual template was first obtained from the corresponding 
chromosomal representation as described above. It then had 
to be evaluated for fitness. 

It is first necessary to define what is meant by a good 
template for a given image of an object. The basic require- 
ment is that the template should achieve a good match to the 60 
image data of a given test image as well as achieving a 
geometric fit with small deformation. The fitting process 
used to test the fitness of a given template will correspond 
to the fitting process to be used on the final composite 
template when matching test images. 65 

Given the above requirements it can be seen that a trivial 
solution for obtaining a maximal fit and image response is a 



template which saturates a test image with all possible, 
feature types to be matched. Since it is also required that a 
template description is as concise as possible, that is without 
an instances excess of feature types, a cost is associated with 
the template which increases with increasing numbers of 
occurrences of feature types and so prevents oversaturation 
of template nodes. 

Bringing these elements together one can formulate an 
objective measure of merit for a given template applied to a 
given test image where greater values indicate more desir- 
able templates subject to our criteria. The merit equation 
used in this embodiment was the merit value G given by: 
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where the variables are: 



G 


unnormalised number of merit 


n 


number of nodes 


N 


desirable number of nodes 




image response representation types t 


U 


type of node i 




position of feature type i 


num_cdges 


number of inter-node connections 




current distance between connected nodes 




original distance between connected nodes 


G. 


Global mesh scale 


P 


excessive node penalty constant 


ki 


empirical determined weighting factors 



The final template had to work well for most of the novel 
faces likely to be encoimtered so it was necessary to assess 
the template's merit measure against a range of faces. An 
environment of face images was therefore used which is 
separate from the initial training set. To evaluate each 
member of the population at a given generation a sample of 
test face images was randomly picked from the environment 
set. Each individual template was then fitted to each of the 
test set of images using a small number of iterations on the 
above measure of merit and a single average merit value 
calculated per template. These values formed the final 
unnormalised measures of merit for each template. 

The merit value G described above includes both negative 
and positive values. It is necessary to form a measure of 
fitness for use by the genetic algorithm which is both 
non-negative and exhibits an even spread of values across 
the population during any given generation so the better 
members can be effectively discriminated for. The measure 
G was normalised to achieve these criteria and yield a fitness 
value that was useable. 

There are several possibilities for the normalisation of the 
objective merit measure to form a more useful fitness 
measure. Normalisation was performed such that an average 
performing member of the population received a fitness of 
unity, with better members having greater fitness values and 
worse members having lesser but always non-negative fit- 
ness values. A possible normalisation scheme is to calculate 
the normalised fitness f using the equation. 



/ = ! + 



(C-P) 



where the mean and standard deviations a of the values G 
across the population are first calculated and then the 
normalised fitness is calculated for each population member, 
"k** is a small constant, typically less than 3, which governs 
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the spread of resulting fitness values. In a few instances 
where the resulting fitness f will be negative, it is set to 0. 

Genetic algorithms select members from the existing 
population to breed and produce the next generation. This 
selection is performed in a probabilistic fashion depending 5 
on the relative fitness of each member. The actual method 
used in this embodiment was stochastic remainder selection 
which has been found to perform better than purely random 
procedures. This method guarantees that the better than 
average members will be appropriately represented in the 
succeeding generation by determinislically assigning part of 
the breeding population prior to random selection of the 
remainder of the breeding population. 

Gossover is the most powerful of the genetic operators 
employed in this genetic algorithm and will now be 
explained with references to FIGS. 11 and 12. Crossover 
provides the means for useful attributes of one individual to 
combine with other useful attributes from a second indi- 
vidual to produce a child which hopefully exhibits the 
strengths of both. Many combinations are tried together with 
poor combinations being weeded out by fitness selection. 
Crossover was applied with a high probability of about 0.6. 

To effect crossover, two cut points 70 and 71 were 
randomly chosen using a uniform probability distribution 
over the length of two chromosomes G* and G^ having gene 
strings G^i to G*o and G^i to G^^ respectively. The cut ^ 
points 70 and 71 define two regions of the parent chromo- 
somes 0^*0 to G'*^. and G^^ to G*^ respectively. These were 
exchanged to create two children chromosomes G*^ and G^ 
as shown in FIG. 12. 

There was a minor problem which had to be resolved to 
maintain the integrity of both child chromosomes. It would 
often happen that the set of zones represented by the 
incoming genes for one child would not match those coded 
for by the outgoing genes as the gene sequences varied. In 
such a case some zones will be multiply represented and 
others completely unrepresented. This was remedied by 
matching surplus genes in the first child G^ outside the 
crossover region with surplus genes in the second child G^ 
also outside the crossover region and exchanging these 
between the two strings. In this way both children G^, G^ 
ended up with a fiill complement of genes representing all 
the possible tokens zone position as we require. The integ- 
rity of the sequence of genes in the crossover region was 
always maintained. This type of approach had been success- 
fully applied previously to similar problems where gene 
ordering can vary such as genetic algorithms applied to the 
travelling salesman problem. 

A fiirther technique employed in the genetic algorithm of 
the present invention was mutation, a genetic operator which 
acts on a single gene to randomly change it in some way. The 
mutation operation was applied with a very low probability 
as a background operator to the algorithm as a whole, its 
function being to reintroduce genetic variation which may 
have been lost, or new useful variations. The principal 
mechanisms of this genetic algorithm is the combination of 
the crossover operator and the survival of the fittest selec- 
tion. 

As mutations occur relatively infrequently we adopted the 
following strategy rather than keep checking to see whether 
a mutation will occur for every gene: we decide in advance 
which genes will mutate. Assuming probability p of single 
gene mutating, the cumulative probability c of a single 
mutation in a gene string is given by: 

C-l-(l-p)« 

It was therefore possible to randomly select a cumulative 
probability c in the range (0, 1) and calculate the next gene 
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to mutate as being the gene n gene-positions ahead in the 
gene string. The value of n was calculated by rearranging the 
above equation to give: 

n-tog(l-c/tog(l-p)) 

When the mutation occurred within a gene in the model, 
the activity of the gene and hence the corresponding token 
was toggled. If the gene was active and so contained a token 
it was marked inactive, otherwise it was marked active and 
the corresponding feature type field F3 assigned a random 
feature type from those allowable. As there was a gene for 
every zone within an image, a feature type could, potentially, 
have become active with any zone Z, at any of the available 
image locations. 

The inversion operator was a further type of mutation 
operator also included in the algorithm of the present 
invention which operated at a fairly low probability. It 
operates on a single chromosome and its affect is to com- 
pletely invert the sequence order of a randomly chosen 
string of genes on the chromosome as shown in FIG. 13. 

The gene string of a chromosome G^ to be inverted is 
chosen by randomly selecting two end points 72, 73 based 
on the uniform probability function over the entire length of 
the chromosome. This gene string is then reversed in order 
within the chromosome G* to form a new chromosome G^. 

It is to be noted that inversion does not affect the template 
corresponding to the chromosome during the cunrent gen- 
eration as the same tokens are coded regardless of the gene 
order. The main effect of inversion is to allow genes to 
change position on the chromosome such that co-adapted 
genes tend to become clustered over time. Such clusters of 
genes will tend to be preserved during crossover as the cut 
point is less likely to fall between co-adapted genes if they 
are physically close on the chromosome. As an example, 
genes coding for feature types good at matching an eye may 
cluster and then usefully propagate through the population 
and eventually be combined with a smaller co-adapted gene 
cluster relevant to matching a mouth to thereby improve the 
template as a whole. 

TTie algorithm described above was carried out on a 
Meiko transputer array consisting of an array of 64 indi- 
vidual transputer based processing nodes and some special 
nodes consisting of graphics display cards. The actual imple- 
mentation of the algorithm described above is arbitrary to 
the extent that it only affects the time taken to compute the 
generalised template from the set of individual templates. 

We claim: 

1. A method of forming a template of an image of an 
object comprising steps of: 

a) detecting occurrences of at least one feature type within 
the image which meet a repective criterion; 

b) for each such occurrence, determining a feature posi- 
tion which is a point within the image at which the 
occurrence of the feature type is to be considered as 
being located; and 

c) constructing a structural mesh of links between the 
feature positions. 

2. A method as claimed in claim 1 in which step a 
includes, in respect of each feature type the steps of: 

d) forming a representation image in which only those 
portions of the image which are of the feature type are 
enhanced; 

e) thresholding the representation image to obtain an 
image bitmap; 

£) locating areas of activity within the image bitmap; and 
g) determining which of the located areas of activity meet 

the criterion to be detected as an occurrence of the 

feature type. 
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3. A method as claimed in claim 2 in which the step of 
locating areas of activity within an image bit map includes 
comparing the corresponding picture elements of the image 
bit maps of different orientations of a given feature type and 
retaining only the maximum valued picture elements. 5 

4. A method as claimed in claim 2 in which the criterion 
is that the area of activity must be greater than a threshold 
size. 

5. A method as claimed in claim 2 in which the feature 
position is the centroid of the corresponding area of activity lO 
in a modified image bitmap. 

6. A method as claimed in claim 1 in which a structural 
mesh is constructed by Delauney triangulation. 
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7. A method as claimed in claim 1 in which the template 
is modified by deleting lengths greater than a predetermined 
length. 

8. A method as claimed in claim 1 in which the feature 
types are selected from a set of feature type which includes 
oriented edges, oriented peaks and oriented valleys. 

9. A method of forming a template comprising deriving a 
plurality of templates forming respective images of different 
members of the class of objects according to claim 1 and 
combining the templates to form a generalised template. 

10. A method as claimed in claim 9 in which the templates 
are combined using a genetic algorithm. 

* 4e * * * 
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